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(57) Abstract: 

PURPOSE: To provide an information input device as 
a human interface excellent in handleabi 1 i ty by 
integrating pen input and speech input. 
CONSTITUTION: This information input device is 
equipped with an input pen 1 which has a microphone 
la and a function for sending a speech signal 
gathered by the microphone la to a computer body 
through a cord or by radio, a display and tablet 2 
which detects a touch by the input pen 1, means 5, 
6, and 7 which performs a speech recognizing 
process and a natural language process for the 
speech signal from the microphone la, and means 8, 
9, 10, and 11 which performs a character 
recognizing process, an image recognizing process, 
and a position recognizing process for the input 
signal from the table 2. Further, this device is 
equipped with a means 15 which combines the input 
from the microphone la and the input from the 
tablet 2 together and sends data or a command to an 
operating system 18 or application program 19. 
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(54) INFORMATION INPUT DEVICE 

(57)Abstract: 

PURPOSE: To provide an information input device as a 
human interface excellent in handleability by integrating pen 
input and speech input. 

CONSTITUTION: This information input device is equipped 
with an input pen 1 which has a microphone la and a 
function for sending a speech signal gathered by the 
microphone 1 a to a computer body through a cord or by 
radio, a display and tablet 2 which detects a touch by the 
input pen 1 , means 5. 6. and 7 which performs a speech 
recognizing process and a natural language process for the 
speech signal from the microphone la. and means 8. 9, 10, 
and 1 1 which performs a character recognizing process, an 
image recognizing process, and a position recognizing 
process for the input signal from the table 2, Further, this 
device is equipped with a means 15 which combines the 
input from the microphone la and the input from the tablet 
2 together and sends data or a command to an operating 
system 1 8 or application program 1 9. 




LEGAL STATUS 

[Date of request for examination] 

[Date of sending the examiner s decision of 
rejection] 

[Kind of final disposal of application other than 
the examiner's decision of rejection or 
application converted registration] 

[Date of final disposal for application] 

[Patent number] 

[Date of registration] 

[Number of appeal against examiner's decision of 



http://wwwl 9apdl.ncipi.goop/PAl /result^detail/main/wAAAzpaqRn^ 1311 08P 1 .htm 1 2/1 4/2004 



Pagel ofl 



* NOTICES * 




JPO 6Uid NCXPI are not responsible £or any 
damages caused by the use o£ this translation. 

1 .This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2.**** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



CLAIMS 
[Claim(s)] 

[Claim 1] An input pen with the function to transmit the sound signal extracted with a microphone and 
this microphone to the body of a computer by the cable or wireless, The tablet of the display 
combination which detects contact of this input pen, and a means to perform speech recognition 
processing and natural language processing to the sound signal from said microphone, A means to 
perform character recognition processing, image recognition processing, and location recognition 
processing to the input signal from said tablet. The information input unit characterized by having a 
means to give data or a command to an operating system or an application program combining the input 
from said microphone, and the input from said tablet. 

[Claim 2] The information input unit according to claim 1 characterized by adopting the higher one of 
whenever [ recognition ] as the data given to an operating system or an application program, or a 
command when the input from a microphone and the input from a tablet were inputs about tiie same 
object. 

[Claim 3] The information input unit according to claim 1 or 2 which was made to input data from a 
microphone and was characterized by making editing commands, such as insertion and deletion, input 
by gesture with the input pen. 

[Claim 4] The information input unit according to claim 1, 2, or 3 characterized by having the memory 
which accumulates the sound signal extracted with the microphone in an input pen. 
[Claim 5] The information input unit according to claim 1, 2, 3, or 4 characterized by transmitting a 
sound signal from the body of a computer to an input pen, and reproducing sound with a microphone. 



[Translation done.] 
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DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Industrial Application] This invention relates to an information input imit for a user to input 
information, such as data and a command, in a personal computer, a word processor, etc. 
[0002] 

[Description of the Prior Art] As this kind of an information input device, although the keyboard is used 
from ancient times, recently, a tablet top is traced with a pen and the type which inputs an alphabetic 
character and a graphic form is also beginning to be used. Moreover, what is depended on voice input is 
realizable by improvement in a speech recognition technique. 
[0003] 

[Problem(s) to be Solved by the Invention] Although various things were appearing as an informational 
input gestalt as mentioned above, it was used independently conventionally, respectively. For example, 
even if it was equipment equipped with the pen input function and the voice input function, a pen input 
and voice input had been independent and the item inputted with a pen and the item inputted with voice 
were distinguished. Moreover, a mutual input was not related directly. 

[0004] Since the microphone for inputting voice is attached in the body of a computer on the other hand 
or it connected through the code, while wiring became obstructive, it was able to be used only near the 
body of a computer. Moreover, the power source of the body of a computer was switched on, and if it 
had not changed into the condition which can accept audio, voice input from a microphone was not able 
to be used. 

[0005] The place which this invention is proposed in view of the above-mentioned point, and is made 
into the purpose unites a pen input and voice input, and is to offer the information input unit as a user- 
friendly hxmian interface. 
[0006] 

[Means for Solving the Problem] An input pen with the function to transmit the sound signal extracted 
with a microphone and this microphone to the body of a computer by the cable or wireless in order that 
this invention may attain the above-mentioned purpose. The tablet of the display combination which 
detects contact of this input pen, and a means to perform speech recognition processing and natural 
language processing to the sound signal from said microphone, A means to perform character 
recognition processing, image recognition processing, and location recognition processing to the input 
signal from said tablet. He is trying to have a means to give data or a command to an operating system 
or an application program combining the input from said microphone, and the input from said tablet. 
[0007] For example, when the input from a microphone and the input from a tablet are inputs about the 
same object, the higher one of whenever [ recognition ] is adopted as the data given to an operating 
system or an application program, or a command. Moreover, data are made to input from a microphone 
and it is made to make editing commands, such as insertion and deletion, input gesture with an input 
pen. 

[0008] It can have the memory which, on the other hand, accumulates the soimd signal extracted with 
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the microphone in an input p^^^ 

[0009] Furthermore, a sound signal is transmitted from the body of a computer to an input pen, and 

sound can be reproduced with a microphone. 

[0010] 

[Function] If it is in the information input unit of this invention and a user talks toward the input pen 
which it had in the hand, the voice will be extracted with the microphone built in the input pen, and will 
be transmitted to the body of a computer by the cable or wireless. Moreover, the information will be 
inputted if the tablet of display combination is traced with an input pen. 

[001 1] Inside the body of a computer, while speech recognition processing and natural language 
processing are performed to the sound signal from the microphone of an input pen, character recognition 
processing, image recognition processing, and location recognition processing are performed to the input 
signal from a tablet. 

[0012] And the input from a microphone and the input from a tablet are combined appropriately, and are 
given as data or a command to an operating system or an application program. For example, when the 
input from a microphone and the input from a tablet are inputs about the same object, the higher one of 
whenever [ recognition ] is adopted and it is given as data or a conmiand. Moreover, a user can input 
editing commands, such as insertion and deletion, with an input pen to the data inputted from the 
microphone. 

[0013] On the other hand, when it has the memory which accumulates the sound signal extracted with 
the microphone and there is no body of a computer in operating status, also when separating and 
working on the input pen from the body of a computer, voice can be inputted into it, and it can bimdle 
up behind, and a sound signal can be transmitted to the body of a computer. 

[0014] Furthermore, a sound signal is transmitted from the body of a computer to an input pen, and 

sound can be reproduced with a microphone. 

[0015] 

[Example] Hereafter, with reference to a drawing, it explains about the example of this invention. 
[0016] Drawing 1 is the block diagram showing one example of the information input xmit of this 
invention. 

[0017] While this example processes the signal from the input pen 1 which a user has and uses for a 
hand, the tablet 2 of the display combination which presses the input pen 1 and draws an alphabetic 
character, a graphic form, etc., and the input pen 1 and a tablet 2 in drawing 1 Two or more circuit 
sections which give these a signal or control (both hardware and software are included.) It consists of an 
operating system 1 8 which manages the overall control within the body of a computer, and an 
application program 19 which does a predetermined activity. 

[0018] Moreover, microphone la for extracting the voice other than the fimdamental fimction to trace a 
tablet 2 on the input pen 1, and to give the locus as information to it. While transmitting the sound signal 
which was extracted by memory lb which accumulates the soxmd signal extracted by this microphone 
la, and microphone la, or was accumulated by memory lb to the body of a computer Transceiver 
section 1 c which receives the sound signal at the time of using a microphone 1 as a loudspeaker from the 
body of a computer is prepared. In addition, memory lb may be accumulated by which method of digital 
one or an analog. Moreover, it is possible between the input pen 1 , and a receive section 3 and the 
transmitting section 4 not only wireless but to connect with a cable. 

[0019] As two or more circuit sections which give these a signal or control on the other hand while 
processing the signal from the input pen 1 and a tablet 2 The part which divides roughly and 
communicates with the input pen 1 (a receive section 3, transmitting section 4), The part which extracts 
an alphabetic character and a command from the inputted voice (the speech recognition section 5, the 
natural-language-processing section 6, an alphabetic character and a command input area 7), The part 
which extracts an alphabetic character, an image, a location, etc. from the locus of contact of the input 
pen 1 to a tablet 2 (the image-processing section 8, the character recognition section 9, the image 
recognition section 10, the gesture input section 11, the location recognition section 12, the location 
input section 13, image input section 14), It consists of a part (voice generation section 16) which 
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generates the sound signal givefno the input pen 1, a part (display and control section 17) which 
controls the display of a tablet 2, and a part (comprehensive processing section 15) which performs the 
whole control. In addition, the natural-language-processing section 6, and an alphabetic character and a 
conunand input area 7 are used also for the processing to the hand written character inputted from the 
tablet 2. 

[0020] Next, actuation of the above-mentioned example is explained. 

[0021] First, the user shall have started the suitable application program 19, in order to be in the 
condition in which an operating system 1 8 and a dialogue are possible or to carry out desired business. 
An application program 19 is a word processor etc. 

[0022] Although a user inputs data or a command according to the function of an operating system 18 or 
an application program 19, voice input and a pen input can be used as the approach of an input of data or 
a command in this case. In addition, when the keyboard is connected to the body of a computer, 
naturally the input from a keyboard can also be performed. 

[0023] If a user speaks toward the input pen 1, this voice will be extracted by microphone la and will be 
transmitted to the body side of a computer by transceiver section Ic by wireless. In addition, by 
considering as sound recording mode, the sound signal is temporarily accumulated in memory lb, and it 
can transmit to the body side of a computer collectively behind. In this case, as long as there is an input 
pen 1 , it is not necessary to be near the body of a computer at the time of utterance and, and the body of 
a computer may not be in operating status. 

[0024] In the body side of a computer, by the receive section 3, the sound signal from the input pen 1 is 
received, it gets over, and the speech recognition section 5 is passed. 

[0025] In the speech recognition section 5, to the received soimd signal, well-known analysis 
processings (voice section detection, sonagraphy, a feature extraction, segmentation, phoneme 
recognition, etc.) are performed, and the data in which the train of a phoneme is shown are outputted. 
[0026] In the natural-language-processing section 6, to the train of a phoneme, well-known analysis 
processings (word collating, syntax analysis, semantic analysis, etc.) are performed in the field of 
machine translation, and the extracted character string is outputted. 

[0027] In an alphabetic character and a command input area 7, it prepares to a predetermined format by 
making into data or a conmiand the character string given from the natural-language-processing section 
6, and gives the comprehensive processing section 15. 

[0028] In the comprehensive processing section 15, the data or the command given from the alphabetic 
character and the command input area 7 is given to an operating system 18, and if an operating system 
18 is a command to self, it will accept it, and it hands over the thing to an application program 19 to an 
application program 19. 

[0029] On the other hand, when a user traces a tablet 2 top and draws an alphabetic character, a graphic 
form, etc. with the input pen 1 , the condition of the contact to a tablet 2 is detected by the image- 
processing section 8. 

[0030] In the character recognition section 9, when it distinguishes whether what was drawn is an 
alphabetic character and there is an alphabetic character which can be recognized about it, the alphabetic 
character recognized in the natural-language-processing section 6 is passed. Actuation of the natural- 
language-processing section 6 to henceforth is the same as that of the case of voice input. 
[0031] Moreover, in the image recognition section 10, when it distinguishes whether it is the gesture 
(motion of the pen specified corresponding to predetermined commands, such as insertion and deletion.) 
which was drawn on the tablet 2 and there is gesture which can be recognized about it, it is told to the 
gesture input section 1 1 . 

[0032] In the gesture input section 11, the data or the command corresponding to gesture is given to the 
comprehensive processing section 15. 

[0033] In the comprehensive processing section 15, the data or the command given from the gesture 
input section 7 is given to an operating system 18, and if an operating system 18 is the data and the 
command to an application program 19, it will be handed over to an application program 19. 
[0034] Moreover, in the location recognition section 12, the contact location of the input pen on a tablet 
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2 is recognized, and the infornTSfTon is given to the location input section 
[0035] The location input section 13 prepares the data in which a location is shown to a predetermined 
format, and passes them to the comprehensive processing section 15. 

[0036] In the comprehensive processing section 15, the data given from the location input section 13 are 
given to an operating system 18, and if an operating system 18 is data to an application program 19, it 
will be handed over to an application program 19. 

[0037] Moreover, the image input section 14 is given to the comprehensive processing section 15 by 
making into image data the locus itself drawn on the tablet 2, and the comprehensive processing section 
15 is given to an operating system 18, and it hands it over to an application program 19 about the data to 
an application program 19. 

[0038] On the other hand, when operating microphone la of the input pen 1 as a loudspeaker, the 
comprehensive processing section 15 gives audio data to the voice generation section 16. 
[0039] In the voice generation section 16, speech synthesis etc. is performed, a sound signal is 
generated, it gives the transmitting section 4, and the transmitting section 4 transmits to the input pen 1 
by wireless. 

[0040] In the input pen 1 side, if transceiver section Ic receives this, it will get over, will consider as a 
sound signal, it will give microphone la, and will change into sound. 

[0041] Moreover, a display and control section 17 controls the contents displayed on the basis of control 
of the comprehensive processing section 15, and the tablet 2 used as display combination. 
[0042] Although basic actuation of voice input and a pen input has been explained until now, in fact, 
both inputs are related closely mutually and performed. Namely, the comprehensive processing section 
15 combines appropriately the input from microphone la of the input pen 1, and the input from a tablet 
2, and gives them to an operating system 1 8 and an application program 19 as data or a conmiand. 
[0043] For example, when the input from microphone 1 a and the input from a tablet 2 are inputs about 
the same object, the comprehensive processing section 15 adopts the higher one of whenever 
[ recognition ] as the data given to an operating system 18 or an application program 19, or a command 
(when the data input of an item with an application program 19 is being demanded as an example etc.). 
In addition, whenever [ recognition ] is a value extractable in the process of processing of the 
recognition in the speech recognition section 5, the natural-language-processing section 6, the character 
recognition section 9, and image recognition section 10 grade. By doing in this way, the precision of 
recognition can be raised sharply. 

[0044] Moreover, a text etc. can be smoothly inputted by making editing commands, such as insertion 
and deletion, input by gesture with the input pen 1 and a tablet 2, inputting data from microphone la. 
[0045] For example, although "BTA" was inputted with voice from microphone la, the input pen 1 
performs the gesture of the deletion which draws "x" on a mistaken part "B" on a tablet 2 to input 
"DATA" (**) and originally, as shown in drawing 2 (**). 

[0046] Thereby, "B" is eliminated and "TA" of one character is got blocked in front (**). 

[0047] Subsequently, it will be set to "DATA", if the gesture "**" of insertion is performed and "DA" is 

inputted with voice from (**) and microphone la before "TA" (**). 

[0048] In addition, it supposes that a back alphabetic character remains as it is, and when an alphabetic 
character is newly inserted, you may make it a back alphabetic character shift only by making the 
deleted alphabetic character into a null if needed, although he was trying to get a back alphabetic 
character blocked with drawing 2 in front in the case of deletion, 
[0049] 

[Effect of the Invention] If it is in the information input unit of this invention as explained above, there 
is the following effectiveness. 

[0050] ** Since the input of data or a command can be performed combining voice input and a pen 
input appropriately, the user-friendliness as a human interface becomes very good. 
[0051] ** Even if it makes it voice input and makes it a pen input, there is an inadequate point in respect 
of recognition precision, but since whenever [ recognition ] adopts the higher one when there is an input 
from both sides about the same object, recognition precision can be raised sharply. 
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[0052] ** Since editing commfffls, such as insertion and deletion, can be inputted by the gesture of an 

input pen, inputting data with voice, smooth document preparation etc. can be performed. 

[0053] ** Since the microphone and the body of a computer of an input pen are connectable on radio, 

there is no obstructive wiring and it can consider as comfortable work environment. 

[0054] ** If the input pen is carried when there is no body of a computer in operating status, or when 

separating and working from the body of a computer since the memory which accimiulates a soimd 

signal is prepared in the input pen, it is not necessary to input document data etc. on that spot, to bundle 

up behind, to transmit to the body of a computer, and to forget the idea thought of. 



[Translation done.] 
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* NOTICES * ^ ^ 

JPO and NCIPI are not responsible for €Uiy 
damages caused by the use of this translation. 

1 .This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2.**** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



DRAWINGS 



[Drawin g 2] 
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[Drawing 1] 
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http://www4apdl.ncipi.gojp/cgi-bin/tran_web_cgi_ejje 



12/14/2004 



This Page is Inserted by IFW Indexing and Scanning 
Operations and is not part of the Official Record 



Defective images within this document are accurate representations of the original 
documents submitted by the applicant. 

Defects in the images include but are not limited to the items checked: 

□ BLACK BORDERS 

□ IMAGE CUT OFF AT TOP, BOTTOM OR SIDES 



U FADED TEXT OR DRAWING 

□ BLURRED OR ILLEGIBLE TEXT OR DRAWING 

□ SKEWED/SLANTED IMAGES 

□ COLOR OR BLACK AND WHITE PHOTOGRAPHS 

□ GRAY SCALE DOCUMENTS . 

□ LINES OR MARKS ON ORIGINAL DOCUMENT 

□ REFERENCE(S) OR EXHIBIT(S) SUBMITTED ARE POOR QUALITY 

□ OTHER: [ 

IMAGES ARE BEST AVAILABLE COPY. 
As rescanning these documents will not correct the image 
problems checked, please do not report these problems to 
the IFW Image Problem Mailbox. 



BEST AVAILABLE IMAGES 




